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CLAIMS 

1. An image processing device comprising: 

an image pick-up device having the fixed positional 

relation with a measurement object; 
5 an attitude sensor for measuring the attitude at an 

image pick-up visual point of said image pick-up device; 
storage means for storing the calculation information 

to calculate the attitude and/or position of said measurement 

object on the basis of an output from said attitude sensor; 
10 target image setting means for setting a target image 

that is an object for detecting a predetermined index on 

the basis of a picked-up image picked up by said image pick-up 

device; 

detecting means for detecting the position of said index 
15 in said target image by performing a template matching 
process between a template image of said index and said target 
image ; 

updating means for updating said calculation 
information stored in said storage means on the basis of 
20 a detected position of said index detected by said detecting 
means ; and 

calculation means for calculating the attitude and/or 
position of said measurement object on the basis of said 
measured value and said calculation information updated by 
25 said updating means. 
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2. The image processing device according to claim 1, 
wherein said target image setting means obtains a prediction 
position of the index in said picked-up image employing said 
measured value and said calculation information stored in 

5 said storage means , creates an image with a peripheral area 
around said prediction position in said picked-up image 
subjected to a rotational process on the basis of a rotational 
angle in a roll direction of said image pick-up device derived 
from said measured value , and outputs said image as a target 
10 image. 

3. The image processing device according to claim 2, 
wherein said calculation information is the correction 
information to correct for an error in the measured value 

15 of attitude measured by said attitude sensor, and said 
calculation means calculates the attitude of said 
measurement object on the basis of said measured value and 
said correction information. 

20 4. The image processing device according to claim 1, 
wherein said calculation information is the correction 
information to correct for an error in the measured value 
of attitude measured by said attitude sensor, and said 
calculation means calculates the attitude of said 

25 measurement object on the basis of said measured value and 
said correction information. 
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5. The image processing device according to claim 1 or 
2, wherein said calculation information is the correction 
information to correct for an error in the measured value 
of attitude measured by said attitude sensor and the position 

5 information of the image pick-up visual point of said image 
pickup device, and said calculation means calculates the 
position and attitude of said measurement object on the basis 
of said measured value, said correction information and said 
position information. 

10 

6. The image processing device according to claim 1 or 
2, wherein said calculation information is the position 
information of the image pick-up visual point of said image 
pickup device, and said calculation means calculates the 

15 position and attitude of said measurement object on the basis 
of said measured value and said position inf ormation . 

7. The image processing device according to claim 5 or 
6, wherein said updating means updates the position 

20 information in the two directions except for a depth 

direction in the camera coordinate system of said image 
pickup device, even when an index of only a single point 
is detected in said detecting means, 

25 8. The image processing device according to any one of 
claims 3 to 5, wherein said correction information is the 
information to correct for an error in the azimuth direction 



among the measured values of the attitude measured by said 
attitude sensor. 



9. The image processing device according to any one of 
claims 1 to 8, wherein said updating means updates said 
calculation information on the basis of the detected position 
of said index in said picked-up image. 

10. The image processing device according to claim 3, 4 
or 6 , wherein said updating means updates said calculation 
information on the basis of a typical value of the updated 
value of said calculation information obtained for each index 
when a plurality of indices are detected in said detecting 
means . 



11. The image processing device according to claim 3, 
wherein said updating means updates said calculation 
information on the basis of a dislocation between the 
prediction position and said detected position of said index 
in said target image. 



12. The image processing device according to claim 11, 
wherein said updating means updates said calculation 
information on the basis of a typical value of said 
dislocation obtained for each index when a plurality of 
indices are detected in said detecting means. 



- 46 - 



13. The image processing device according to any one of 
claims 5 to 7, wherein said updating means updates the 
position information in three directions in the camera 
coordinate system of said image pick-up device, when two 

5 or more indices are detected in said detecting means. 

14. The image processing device according to any one of 
claims 1 to 13, wherein said index is a projected image of 
a landmark in the real space onto said picked-up image. 

10 

15. The image processing device according to claim 14, 
wherein the position of said landmark in the real space is 
known . 

15 16. The image processing device according to claim 4, 11 
or 12 , wherein said index is an image feature on saidpicked-up 
image . 

17. The image processing device according to any one of 
20 claims 1 to 16, wherein said measurement object is an image 

pick-up visual point of said image pick-up device. 

18. The image processing device according to claim 17, 
further comprising display means for displaying said 

25 picked- up image with the image in the virtual space 

superposed thereon on the basis of the attitude, or position 
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and attitude of said image pick-up device calculated by said 
calculation means . 

19. The image processing device according to any one of 
5 claims 1 to 16, wherein said measurement object is a visual 

point of the observer, and said image processing device 
further comprises display means for displaying the image 
in the virtual space drawn on the basis of the attitude or 
position and attitude of the observer calculated by said 
10 calculation means on said display screen, while optically 

transmitting the image in the real space through said display . 
screen observed by the observer. 

20. An image processing device in which the position of 
15 an index in a picked-up image picked up by an image pick-up 

device is detected by template matching employing a template 
image of said index, comprising: 

an attitude sensor for measuring the attitude at an 
image pick-up visual point of said image pick-up device; 

20 target image creating means for creating a target image 

having a peripheral area around a prediction position in 
said picked-up image subjected to a rotational process on 
the basis of the rotational angle in a roll direction from 
said measured values by obtaining said prediction position 

25 of the index in said picked-up image, employing the measured 
value of said attitude measured by said attitude sensor, 
and outputting the target image; and 
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detecting means for detecting the position of said index 
in said picked-up image by performing a template matching 
process between said template image and said target image. 

5 21. The image processing device according to any one of 
claims 1 to 20, further comprising template image creating 
means for creating a template image for detecting said index 
from the image, wherein said template image creating means 
specifies the position of the index in the picked-up image 

10 when said image pickup device is located at a predetermined 
position and attitude and creates a template image of the 
index with a peripheral area around the position of the index 
in said picked-up image subjected to a rotational process 
on the basis of the rotational angle in a roll direction 

15 from said predetermined positional attitude. 

22. The image processing device according to any one of 
claims 1 to 21, wherein said attitude sensor measures the 
attitude of an image pick-up visual point of said image 

20 pick-up device in a state where there is an accumulated error 
in the measured value in the azimuth direction. 

23. The image processing device according to claim 22, 
wherein said attitude sensor is constituted of a gyro sensor . 

25 

24. An image processing method comprising: 
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an image pick-up step of picking up an image with an 
image pick-up device having the fixed positional relation 
with a measurement object; 

an attitude measuring step of measuring the attitude 
5 at an image pick-up visual point of said image pick-up device ; 

a storage step of storing the calculation information 
to calculate the attitude and/or position of said measurement 
object on the basis of the measured value measured at said 
attitude measuring step; 
10 a target image setting step of setting a target image 

that is an object for detecting a predetermined index on 
the basis of said picked -up image; 

a detecting step of detecting the position of said index 
in said target image by performing a template matching 
15 process between a template image of said index and said target 
image ; 

an updating step of updating said calculation 
information stored at said storage step on the basis of a 
detected position of said index detected at said detecting 
20 step; and 

a calculating step of calculating the attitude and/or 
position of said measurement object on the basis of said 
measured value and said calculation information updated at 
said updating step. 

25 



25. The image processing method according to claim 23, 
wherein said target image setting step comprises creating 
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a target image with a peripheral area around a prediction 
position in said picked-up image subjected to a rotational 
process on the basis of the rotational angle in a roll 
direction of said image pick-up device derived from said 
5 measured value by obtaining said prediction position of the 
index in said picked-up image, employing the measured value 
and the calculation information stored at said storage step . 

26 . A program code for executing the image processing method 
10 according to claim 24 or 25. 

27. A storage medium storing the program code according 
to claim 26. 



